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Digital Fingerprints and Watermarks for Images 



This invention relates to the q>plication of digital fingerprints and/or 
watermarks to images. 

5 Video fingerprinting can be used for video content identification and 

associated applications. It is desirable that tibte video fingerprint recognition can proceed 
successfidly despite certain distortions to the video content, such as compression, scaling, or 
translation. 

Background 

10 A typical video fingetprint extraction scheme consists of a first stage of 

extracting features fi-om video fi^mes. Many different features have been proposed for use in 
generating fingerprints including: DC (mean luminance) values, luminance and/or 
chrominance histograms, and edges. A common aspect of these schemes is that images are 
usually divided up into regions (e.g. rectangular blocks) and features extracted for each 

IS region. Such a fingerprinting scheme is disclosed in International Patent Application 
WO-A-02/065782. 

The division of an image fi»ne into regions causes recognition problems if the 
images undergo scaling or tcanslational shifts. The fixed grid used to divide up the image 
results in image regions containing different content for a scaled/translated image than for the 
20 unscaled/unshifted original. This causes differences in the resulting fingerprints that will 
often prevent the scaled/translated content fi:om being recognised. 

It is an object of the present invention to address the above mentioned 

25 disadvantages. 

According to a fidrst aspect of the present invention a melhod of 
sqpplying/detecting a digital fingerprint to/in an image comprises: 

dividing the image into regions, the location and size of tiie regions being 
based on a content of the image; and 
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^pl3dng/detectmg Ihe digital fingeiprixit 

According to a second aspect of the present invention amethod of dividing an 
image into regions for the appKcation/detection of a digital fingerprint comprises selecting 
the location and size of the regions according to a content of the image. 

Preferably, the image is divided based on tiie location of a centroid of the 

image. Preferably, a grid for dividing the image is centred substantially at a location of the 
centroid of the image. 

Preferably, horizontal and vertical sizes of divisions of the grid are 
independent of one another. 

Preferably, the size of horizontal and/or vertical divisions is based on a content 
of portions of the image, preferably based on centroids of portions of the image. 

The method advantageously aUows a grid location and size to be calculated 
based on image content, thereby allowing scaling and translation of flie image without 
affecting the watermarking thereof. 

The regions may overly, either horizontaDy and/or vertically. It has been 
found tiiat overlapping regions can be used with advantageously better results. 

According to a third aspect of the invention a mefliod of embedding^detecting 
a digital watermark into an image comprises selecting a size and/or location of a watermark 
pattern based on a content of the image, and flying/detecting flie watermarlc 

Preferably, a location of flie watennaric, which is preferably present as a grid 
element is chosen according to a location of a centroid of ti»e image. Preferably, a size of flie 
watermark, preferably a size of grid for flie watermaric present as grid elements or in grid 
elements, is chosen accordmg to a content of portions of flie image, preferably centroids of 
portions of the image. 

According to a fourth aspect of flie invention fliere is provided a computing 
deviceprogrammed to operate flie mefliod of tiie first and/or second aspect 

According to a fifth aspect of flie invention a computer program product is 
operable to perform flie metiiod of flie first and/or flie second aspect of flie present invention. 

All of flie features described herein may be combined wifli any of flie above 
aspects, in any combination. 
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For a better understanding of liie invention and to show how the same may be 
brought into effect, specific embodiments will now be described, by way of exanqile, with 
reference to the accompanying drawings, in which: 

Figure 1 is a schematic diagram of an image frame showing a centroid; 
S Figure 2 is an image having a grid of blocks ^lied thereto; 

Figure 3 is the image of Figure 2,after horizontal scaling and vertical shifting; 
Figure 4 shows bit error rates (BER) for fingerprints of a first example video 
clip and the same vide clip after scaling/shifting; 

Figure 5 shows BER for a second example video clip after scaling/shifting; 
10 Figure 6 shows a sample image with a "subtifle" ^jplied; 

Figure 7 shows BER for the first example image with a subtitle added; and 
Figure 8 shows BER for the second example image with a subtitle added 



15 The invention is a technique of deriving fingerprints for image/video content 

that improves the recognition rate in cases where the content is scaled or translated. It is also 
described how the same idea can be applied to watermarking in order to help achieve 
detection in scaled/shifled versions of a watermadced image. 

The key idea in this invention disclosure is to divide images into regions in a 

20 manner dependent upon the image content itself In other words, the grid used to divide 

fiwnes into regions should be determiaed loom the image content In this way, the grid can be 
made to scale and shift identically to the image content, resulting m fingerprints for 
scaled/shifi:ed images that are as close as possible to those of the original images. 

A content dependent division of the image can be achieved using centroid 

25 (centre of mass) calculations. The centroid of an image region with luminance Jix, y) is Ihe 
point (x,y) where: 

* y » y 

Note fhat these can be conqmted as: 

X y X y 
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relatively che^ computation. 

Thecentroidl^stheprc^ofsWflingidenticaUytoft^ 



■*8l>ift 



— _f y 



= x + x^ 



also of scaling identically to the image content 



^ m ^ .^ed and .Med ve^™. of ^ 

Of .do ^f'^^r^^.-^^^^^^.^^^ ^ ^ 
prevents perfect invariance to iip-scaHng and rtiOng. 

^'^»^'*'™-««*»«S«mtapractioet»c<mflrfnedwi1i,pa^ 
c<»«ea.(b.addlngadd«ana,da«.*,acWeveate,„i«d^^ 



♦ 



PHNL030536EPP 



5 08.05.2003 
letain fhe original image dimensions. Scale invariance is still achieved if the padding is made 
with pixels of value zero,' such that they do not contribute in the summations above. 

Despite these imperfections in a practical scheme, it will be seen that the 
centroid shifts and scales sufiBciently accurately to achieve considerable robustness in the 
S resulting fingerprints. 

Usiag centroids to provide a scale/shift robust division of the image will be 
explained for the example of splitting an image into a grid consisting of a number of 
rectangular blocks. This is d^ired, for exan^le, by the fingerprint scheme in 
WO-A-02/06S782 the contents of which are incorporated herein by reference, where fhe DC 
10 diSerencra between these blocks are flien used to derive the fingerprint bits. The process 
consists of fhe following steps: 

Calculate the centroid C(jc, y) of fhe whole image - The grid will be centred 
upon this point, providing robustness to translational shifts of fhe image 
Calculate fhe centroid in the horizontal direction xi^ of all fhe imagp pixels left of C. See 
15 Figure 1. 

Calculate also the centroid Xh^ of all image pixels right of C, fhe centroid 
ydbove in fhe vertical dkection of all the image pixels above C, and fhe ceasb^dyboow of all the 
image pixels below C. 

Define block boundaries at 

20 Horizontal directioa* x ^ , and 3c for i e {0,1,2,..., JV} 

iv N 

Vertical direction: y- ^^ ,and y^^^sd. for /€ {0,1,2,...,M} 

M M 

where a is a multiple chosen to control tiie total size of fhe resulting grid of blocks. For 
example, setting a='2 results in a grid spanning most of fhe origmal image, since the centroids 
of typical image content are usually fairly central. The resulting grid consists of 2Mx2N 
25 blocks. 

In figure 1 Xieft is the centroid in fhe horizontal direction of fhe portion of fhe 
image left of fhe total image centroid C(jc, jO 

Figure 2 shows an exaitple grid of blocks formed using fhe above procedure 
fi>r a=2, and M—N=^3. Figure 3 shows tiie grid formed for the same image scaled horizontally 
30 by a £u;tor of 1.1, and shifted vertically by 20 pixels. It can be seen that the image content of 
each block is similar in both cases, and much more so than if a fixed grid had been used. 
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The above ptoceduie was used to foim blocks from which fingerprints were 
subsequently derived as in WO.A-02/065782. Figures 4 and 5 show the fingeq«int bit eiror 
rales (BER) between fingerprints of the original video content, and fingeiprints from 
^aled/shifred versions of the same content. 

fixed bloclcsofl44x80 pixels are also shown; the improvement of thenewcentroidsch^^ 
IS clear. 

Note ftat fte BER, due to sUfls and up.K=ales a« Iowa fa fte •mterv- video 
cl* in flgu™ 4 fl»n flKV fl« -H,,^. ^ j„ J ^ ^ 

bonter do no. influence fte cenS^d position, ftn, 0^ BER only begins to rise at latter Aifis 

or ,,«cale, where the effict of cmpping begte to take effi«t Note also that tl>e B 

do™«caIing are lower than iteup-scaling, as expected, as no cr,^ 
down-scaling. 

An additional advantage of the presented scheme is that it is rdbm to diffeent 

scale ftcton, in the horizontal and vertical directions, ie. it is rol»m *> aspecttatio ch«^ 
Application to Watermarking 

Scaling and shifting of an image can also prevent detection of a watennark 
conlamed in that image. S^ose, as in many wateimarldng schemes (see WO-A-99/45705 

for example), that the watermarkpatterncanbe formulated asatae of noise samples in the 
spatial domain. Tins watermaik pattern is added to the original image in o^ 
watennarked image. The invention presented above aUows the size and shift of the embedded 

watermaikpattemtobesetbyprK^esoftheimagecontentitself(e.g.centroi^^ The 
^termaik detector can automatically find the ^ropriate scale and shift of the watermark 
pattern m the received image by repeating the procedure. 

mpractice, watermark detection is typically very sensitive to scaling, and the 

accuracy of the scalmgindicatedby(e.g.)ihe centroids may not be great enoughto^^^ 
potion of the watennarlc ll.e invention does, however, provide a first estimate of the scale 

fector.and this can be usedtoreduce the number of scale fectois that needtobesearchedin 

<>^'i-'todetectthewatennark.Tl.ustheinventioncanstiUprovideanadvantageinter^ 
30 reduced computation in the watermark detector. 

The method can be appHed equally well to the plication or detection of a 
fingerprint or watermark. 

Further E^eriments 
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This section d^cribes further e^eiiinents concemiDig the above centroid 
based scheme for improved scale and shift robustness. 
Subtitles 

The addition of a subtitle to a video clip adds areas of bright white pixels to 
. S the bottom of the images. Bxperiments were x^erformed to assess whether this disturbs the 
centroid positions sufficiently to cause a significant BER increase when matfihitig 
fingerprints. 

Subtitle were 'simulated' by adding high density *salt and pepper' noise in 
the area where subtitles would be added An example is shown at Figure 6: 
1 0 This was found to cause a BER of around 7% for the centroid scheme 

compared to aroimd 4% for the original scheme of fixed blocks. The position and size of the 
grid was foimd to be only slightly affected by the addition of the 'subtitle'. The Tnmimal 
impact of the subtitle is simply due to it being only a small fraction of the total number of 
image pixels: in these experiments the subtitle occupied 5% of the image height, and 60% of 
IS the image width, Le. only 3% of pixels. 

Overlapping blocks 

The robustness to shifts and scaling is higjhly dependent upon the size of the 
blocks used m calculating the DC differences. This is true for both the fixed block and 
centroid schemes. In order to increase the effective size of blocks an overlap was introduced 
20 as follows: 

Each block is extended horizontally such that it covers 50% of each block 
either side, and likewise vertically. For blocks at the perimeter of the image, the extension is 
made just to the image edge 

A h arming window is applied to the pixel values of the extended block 
25 The DC is calculated 

The danger of overlapping the blocks is that each pixel contributes to multiple 
fingerprint bits, and this may increase the correlation between thenx If the fingerprint bits 
fixim a given frame are correlated then fliey contain redundancy, i.e. we effectively have 
fewer fingerprint bits, which will lead to higher false positive rates (matching fingerprints 
30 from different content). The above overlap method was found to generate fingerprints which 
still appeared very sdmilar to those of the non-overlapping blocks. From this it seems 
reasonable to assume that the resulting fingerprints will still be sufBciently discriminating 
between different content. 
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The fingerprint matching results for the usual set of attacks are shown at 
7 and 8 for two example video clips (dashed line with stars: fixed non-overl^ing 
blocks, solid line with triangles: centroid scheme with non-overl^g blocks, solid line with 
stars: centroid scheme with overl^ping blocks). It can be seen that the overlap of blocks 
results m similar or slightly improved BER in all cases. 



PHNL030536EPP 

9 08.05^003 

CLAIMS: 



1 . A method of applying/detecting a digital fingerprint to/in an image conqirises: 
dividing the image into regions, the location and/or size of the regions being based on a 
content of Ihe image; and 

applying/detecting Ihe digital fingerprint. 

5 

2. A method of dividing an image into regions for the application of a digital 
fingetprint comprises selecting the location and/or size of the regions according to a content 
of the image. 

10 3. A method of as claimed in claim 1 or claim 2, in which the image is divided 

based on the location of a centroid of Ihe image. 

4. A method as claimed in claim 3, in which a grid for dividing the image is 
centred substantially at a location of the centroid of the image. 

15 

5. A metiiod as claimed in claim 4, ia which horizontal and vertical sizes of 
divisions of the grid are independent of one another. 

6. A method as claimed in claim S, in which the size of horizontal and/or vertical 
20 divisions is based on a content of portions of the image. 

7. A method as claimed in claim 6, in which the size of horizontal and/or vertical 
divisions is based on centtoids of portions of the image. 



25 



8. Amethodof embedding/detecting a digital watermark into/in an image 

comprises selecting a size and/or location for a watermark pattern based on a content of the 
image, and embedding/detecting the watermark. 
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chosen according to a location of a centroid of the 



9. A method as claimed in claim 8, in which a location of the 



08.05.2003 
wateimarkis 



10 



A method as claimed in claim 9, in which the watermaric is present as a grid 

element 



11. A method as claimed in clahn 1 0. in which, a size of the watetmadc or a size 
of gnd for the watemiark is chosen accordmg to a content of portions of the image. 

12. A mefliod as claimed in claim 1 1 , m which a size of the watemiark or a size of 
gnd for the wat«imark is chosen according to centroids of portions of the imagp. 

13. ^"^'"I^tingdeviceprogrammedtooperatethemethodofanyoneof claims 1 
to 12. 



14. A conq,uter program product operable to perform the method of any one of 

claims 1 to 12. 
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ABSTRACT: 



A method of dividing an image into grid regions for the ^Ucation of a 
watemiaik or fingerprint uses a calculation of a centtoid of tiie image to centre tiie grid and 
uses centroids of portions of the image to set the size of tiie grid. 
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